A Ground-Truth Video Dataset for the Development and Evaluation of Vision-based Sense-and-Avoid systems

The importance of vision-based systems for Sense-and-Avoid is increasing nowadays as remotely piloted and autonomous UAVs become part of the non-segregated airspace. The development and evaluation of these systems demand flight scenario images which are expensive and risky to obtain. Currently Augmented Reality techniques allow the compositing of real flight scenario images with 3D aircraft models to produce useful realistic images for system development and benchmarking purposes at a much lower cost and risk. With the techniques presented in this paper, 3D aircraft models are positioned firstly in a simulated 3D scene with controlled illumination and rendering parameters. Realistic simulated images are then obtained using an image processing algorithm which fuses the images obtained from the 3D scene with images from real UAV flights taking into account on board camera vibrations. Since the intruder and camera poses are user-defined, ground truth data is available. These ground truth annotations allow to develop and quantitatively evaluate aircraft detection and tracking algorithms. This paper presents the software developed to create a public dataset of 24 videos together with their annotations and some tracking application results

Computer Vision Based General Object Following for GPS-denied Multirotor Unmanned Vehicles

The motivation of this research is to show that visual based object tracking and following is reliable using a cheap GPS-denied multirotor platform such as the AR Drone 2.0. Our architecture allows the user to specify an object in the image that the robot has to follow from an approximate constant distance. At the current stage of our development, in the event of image tracking loss the system starts to hover and waits for the image tracking recovery or second detection, which requires the usage of odometry measurements for self stabilization. During the following task, our software utilizes the forward-facing camera images and part of the IMU data to calculate the references for the four on-board low-level control loops. To obtain a stronger wind disturbance rejection and an improved navigation performance, a yaw heading reference based on the IMU data is internally kept and updated by our control algorithm. We validate the architecture using an AR Drone 2.0 and the OpenTLD tracker in outdoor suburban areas. The experimental tests have shown robustness against wind perturbations, target occlusion and illumination changes, and the system's capability to track a great variety of objects present on suburban areas, for instance: walking or running people, windows, AC machines, static and moving cars and plants

Toward Visual Autonomous Ship Board Landing of a VTOL UAV

In this paper we tackle the problem of landing a helicopter autonomously on a ship deck, using as the main sensor, an on-board colour camera. To create a test-bed, we first adequately simulate the movement of a ship landing platform on the Sea, for different Sea States, for different ships, randomly and realistically enough. We use a commercial parallel robot to get this movement. Once we had this, we developed an accurate and robust computer vision system to measure the pose of the helipad with respect to the on-board camera. To deal with the noise and the possible fails of the computer vision, a state estimator was created. With all of this, we are now able to develop and test a controller that closes the loop and finish the autonomous landing task

AR drone identification and navigation control at CVG-UPM

This article presents the proposal of the Computer Vision Group to the first phase of the international competition “Concurso de Ingeniería de Control 2012, Control Aut ́onomo del seguimiento de trayectorias de un vehículo cuatrirrotor”. This phase consists mainly of two parts: identifying a model and designing a trajectory controller for the AR Drone quadrotor. For the identification task, two models are proposed: a simplified model that captures only the main dynamics of the quadrotor, and a second model based on the physical laws underlying the AR Drone behavior. The trajectory controller design is based on the simplified model, whereas the physical model is used to tune the controller to attain a certain level of robust stability to model uncertainties. The controller design is simplified by the hypothesis that accurate positions sensors will be available to implement a feedback controller

A General Purpose Configurable Navigation Controller for Micro Aerial Multirotor Vehicles

In this paper, we consider the problem of autonomous navigation of multirotor platforms in GPS-denied environments. The focus of this work is on safe navigation based on unperfect odometry measurements, such as on-board optical flow measurements. The multirotor platform is modeled as a flying object with specific kinematic constraints that must be taken into account in order to obtain successful results. A navigation controller is proposed featuring a set of configurable parameters that allow, for instance, to have a configuration setup for fast trajectory following, and another to soften the control laws and make the vehicle navigation more precise and slow whenever necessary. The proposed controller has been successfully implemented in two different multirotor platforms with similar sensoring capabilities showing the openness and tolerance of the approach. This research is focused around the Computer Vision Group's objective of applying multirotor vehicles to civilian service applications. The presented work was implemented to compete in the International Micro Air Vehicle Conference and Flight Competition IMAV 2012, gaining two awards: the Special Award on "Best Automatic Performance - IMAV 2012" and the second overall prize in the participating category "Indoor Flight Dynamics - Rotary Wing MAV". Most of the code related to the present work is available as two open-source projects hosted in GitHub

A System for the Design and Development of Vision-based Multi-robot Quadrotor Swarms

This paper presents a cost-effective framework for the prototyping of vision-based quadrotor multi-robot systems, which core characteristics are: modularity, compatibility with different platforms and being flight-proven. The framework is fully operative, which is shown in the paper through simulations and real flight tests of up to 5 drones, and was demonstrated with the participation in an international micro-aerial vehicles competition3 where it was awarded with the First Prize in the Indoors Autonomy Challenge. The motivation of this framework is to allow the developers to focus on their own research by decoupling the development of dependent modules, leading to a more cost-effective progress in the project. The basic instance of the framework that we propose, which is flight-proven with the cost-efficient and reliable platform Parrot AR Drone 2.0 and is open-source, includes several modules that can be reused and modified, such as: a basic sequential mission planner, a basic 2D trajectory planner, an odometry state estimator, localization and mapping modules which obtain absolute position measurements using visual markers, a trajectory controller and a visualization module

A Vision-based Quadrotor Swarm for the participation in the 2013 International Micro Air Vehicle Competition

This paper presents a completely autonomous solution to participate in the 2013 International Micro Air Vehicle Indoor Flight Competition (IMAV2013). Our proposal is a modular multi-robot swarm architecture, based on the Robot Operating System (ROS) software framework, where the only information shared among swarm agents is each robot's position. Each swarm agent consists of an AR Drone 2.0 quadrotor connected to a laptop which runs the software architecture. In order to present a completely visual-based solution the localization problem is simplified by the usage of ArUco visual markers. These visual markers are used to sense and map obstacles and to improve the pose estimation based on the IMU and optical data flow by means of an Extended Kalman Filter localization and mapping method. The presented solution and the performance of the CVG UPM team were awarded with the First Prize in the Indoors Autonomy Challenge of the IMAV2013 competition

A System for the Design and Development of Vision-based Multi-robot Quadrotor Swarms

This paper presents a cost-effective framework for the prototyping of vision-based quadrotor multi-robot systems, which core characteristics are: modularity, compatibility with different platforms and being flight-proven. The framework is fully operative, which is shown in the paper through simulations and real flight tests of up to 5 drones, and was demonstrated with the participation in an international micro-aerial vehicles competition3 where it was awarded with the First Prize in the Indoors Autonomy Challenge. The motivation of this framework is to allow the developers to focus on their own research by decoupling the development of dependent modules, leading to a more cost-effective progress in the project. The basic instance of the framework that we propose, which is flight-proven with the cost-efficient and reliable platform Parrot AR Drone 2.0 and is open-source, includes several modules that can be reused and modified, such as: a basic sequential mission planner, a basic 2D trajectory planner, an odometry state estimator, localization and mapping modules which obtain absolute position measurements using visual markers, a trajectory controller and a visualization module

A vision based aerial rbot solution for the IARC 2014 by the Technical University of Madrid

The IARC competitions aim at making the state of the art in UAV progress. The 2014 challenge deals mainly with GPS/Laser denied navigation, Robot-Robot interaction and Obstacle avoidance in the setting of a ground robot herding problem. We present in this paper a drone which will take part in this competition. The platform and hardware it is composed of and the software we designed are introduced. This software has three main components: the visual information acquisition, the mapping algorithm and the Aritificial Intelligence mission planner. A statement of the safety measures integrated in the drone and of our efforts to ensure field testing in conditions as close as possible to the challenge?s is also included

A General Purpose Configurable Controller for Indoors and Outdoors GPS-Denied Navigation for Multirotor Unmanned Aerial Vehicles

This research on odometry based GPS-denied navigation on multirotor Unmanned Aerial Vehicles is focused among the interactions between the odometry sensors and the navigation controller. More precisely, we present a controller architecture that allows to specify a speed specified flight envelope where the quality of the odometry measurements is guaranteed. The controller utilizes a simple point mass kinematic model, described by a set of configurable parameters, to generate a complying speed plan. For experimental testing, we have used down-facing camera optical-flow as odometry measurement. This work is a continuation of prior research to outdoors environments using an AR Drone 2.0 vehicle, as it provides reliable optical flow on a wide range of flying conditions and floor textures. Our experiments show that the architecture is realiable for outdoors flight on altitudes lower than 9 m. A prior version of our code was utilized to compete in the International Micro Air Vehicle Conference and Flight Competition IMAV 2012. The code will be released as an open-source ROS stack hosted on GitHub